Using Prosodic Features of Speech and Audio Localization in Graphical User Interfaces

We describe several approaches for using prosodic features of speech and audio localization to control interactive applications. This information can be used for parameter control, as well as for disambiguating speech recognition. We discuss how characteristics of the spoken sentences can be exploited in the user interface; for example, by considering the speed with which the sentence was spoken and the presence of extraneous utterances. We also show how coarse audio localization can be used for low-fidelity gesture tracking, by inferring the speaker's head position. CR CATEGORIES AND SUBJECT DESCRIPTORS: H.5.2 (User Interfaces): Graphical user interfaces, natural language, voice I/O; I.2.7 (Natural Language Processing). GENERAL TERMS: Design, experimentation, human factors.